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1.  ABSTRACT 


The  mean  search  time  of  observers  looking  for  targets  in  visual 
scenes  with  clutter  is  computed  using  the  Fuzzy  Logic  Approach 
(FLA).  The  FLA  is  presented  by  the  authors  as  a robust  method 
for  the  computation  of  search  times  and  or  probabilities  of 
detection  for  signature  management  decisions  in  any  part  of  the 
electromagnetic  or  acoustic  spectrum.  The  Mamdani/Assilian 
and  Sugeno  models  have  been  investigated  and  are  compared. 

A 44  visual  image  data  set  from  TNO  is  used  to  build  and 
validate  the  fuzzy  logic  model  for  search  time.  The  input 
parameters  are  the:  local  luminance,  range,  aspect,  width, 
wavelet  edge  points  and  the  single  output  is  search  time.  The 
Mamdani/Assilian  model  gave  predicted  mean  search  times 
from  data  not  used  in  the  training  set  that  had  a 0.957  correlation 
to  the  field  search  times.  The  data  set  is  reduced  using  a 
clustering  method  then  modeled  using  the  FLA  and  results  are 
compared  to  experiment. 

Keywords:  Target  acquisition,  fuzzy  logic,  visual  models 


2.  INTRODUCTION 


It  has  been  three  decades  since  Prof.  L.  A.  Zadeh  first  proposed 
fuzzy  set  theory  (logic)  [1],  Following  Mamdani  and  Assilian's 
pioneering  work  in  applying  the  fuzzy  logic  approach  to  a steam 
plant  in  1974  [2],  the  FLA  has  been  finding  a rapidly  growing 
number  of  applications.  These  applications  include, 
transportation  (subways,  helicopters,  elevators,  traffic  control, 
and  air  control  for  highway  tunnels),  automobiles  (engines, 
brakes,  transmission  and  cruise  control  systems),  washing 
machines,  dryers,  refrigerators,  vacuum  cleaners,  TVs,  VCRs, 
video  cameras,  and  other  industries  including  steel,  chemical, 
power  generation,  aerospace,  medical  diagnosis  systems, 
information  technology,  decision  support  and  data  analysis  [3,  4, 
5,  6,  7], 

Although  fuzzy  logic  can  encode  expert  knowledge  directly  and 
easily  using  rules  with  linguistic  labels,  it  usually  takes  some 
time  to  design  and  adjust  the  membership  functions,  which 
quantitatively  define  these  linguistic  labels.  Neural  network 
learning  techniques  can,  in  some  cases,  automate  this  process 
and  substantially  reduce  development  time.  To  enable  a system 
to  deal  with  cognitive  uncertainties  in  a manner  more  like 


humans,  researchers  have  incorporated  the  concept  of  fuzzy 
logic  into  the  neural  network  modeling  approach.  The 
integration  of  these  two  techniques  yields  the  Neuro-Fuzzy 
Approach  (NFA)  [8].  The  NFA  has  potential  to  capture  the 
benefits  of  both  the  fuzzy  and  the  neural  network  methods  into  a 
single  model.  Target  acquisition  models,  based  on  the  theory  of 
signal  detection  or  the  emulation  of  human  early  vision,  are  not 
mature  enough  to  robustly  model,  from  a first  principal  approach 
without  any  laboratory  calibration,  the  human  detection  of 
targets  in  cluttered  scenes.  This  is  because  our  awareness  of  the 
visual  world  is  a result  of  the  perception,  not  merely  detection, 
of  the  spatio-temporal,  spectra-photometric  stimuli  that  is 
transmitted  onto  the  photoreceptors  on  the  retina  [8],  The 
computational  processes  involved  with  perceptual  vision  can  be 
considered  as  the  process  of  linking  generalized  ideas,  such  as 
clutter  or  edge  metrics  flO],  to  retinal  early  vision  data  [9]. 
From  a system  theoretic  point  of  view,  perceptual  vision 
involves  the  mapping  of  early  vision  data  into  one  or  more 
concepts,  and  then  inferring  a meaning  of  the  data  based  on  prior 
experience  and  knowledge.  The  authors  think  that  the  methods 
of  fuzzy  and  neuro-fuzzy  systems  provide  a robust  alternative  to 
complex  models  for  predicting  observed  search  times  and 
detection  probabilities  for  the  vehicles  in  cluttered  scenes  that 
are  typically  modeled  by  defense  department  scientists.  The 
fuzzy  logic  approaches  have  been  used  to  calculate  the  search 
time  of  vehicles  in  different  visual  scenes  within  the 
commercially  available  MATLAB  Fuzzy  Logic  Toolbox. 

3.  FUZZY  MODELS  AND  WAVELETS 

Fuzzy  modeling  of  systems  is  an  approach,  which  describes 
complex  system  behavior,  based  on  fuzzy  logic  with  fuzzy 
predicates  using  a descriptive  language.  Fuzzy  logic  models 
basically  fall  into  two  fundamentally  different  categories,  which 
differ  in  their  ability  to  represent  different  types  of  information. 
The  first  category  includes  linguistic  models  that  are  based  on  a 
collection  of  If-Then  rules  with  vague  predicates  and  use  fuzzy 
reasoning.  One  of  these  reasoning  mechanisms  is  based  on  the 
Mamdani  and  Assilian  fuzzy  inference  method.  Within  this 
method,  a scientist  can  design  the  membership  functions 
manually  and  the  output  membership  functions  are  continuous. 
The  second  method  of  fuzzy  inference  is  based  on  the  Takagi- 
Sugeno-Kang  , or  simply  Sugeno's  method.  In  the  Sugeno 
method  the  membership  functions  are  linear  or  constant. 


Paper  prepared  for  the  RTO  SCI  Workshop  on  “Search  and  Target  Acquisition”,  held  in  Utrecht, 
The  Netherlands,  21-23  June  1999,  and  published  in  RTO  MP-45. 
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For  a review  of  these  methods  as  applied  to  target  acquisition 
modeling  see  [1 1,12]. 


The  method  of  using  wavelets  to  compute  edge  points,  which 
are  then  used  with  fuzzy  logic  to  compute  the  search  time  or  the 
probability  of  detection,  is  derived  from  the  elegant  technique  of 
Mallat  and  Zhong  [15].  In  [15]  a derivation  is  made  of  1-  and  2- 
D wavelet  transforms  using  a smoothing  function.  0(x),  that  is  a 
Gaussian.  The  integral  of  the  function  equals  unity  and  the 
integral  also  converges  to  zero  at  infinity.  We  define  the  first- 
and  second-order  derivative  of  0(x), 


ysa(x)  = 


d6(x) 

dx 


and  y/b(x)  = 


d20(x) 
dx 2 


0) 


By  definition  the  functions  v|;"(x)  and  v|/b(x)  can  be  considered  as 
wavelets  because  their  integral  is  equal  to  zero.  The  following 
subscript  ‘s’  will  be  denoted  as  the  scale  factor. 


Following  standard  methods,  the  wavelet  transform  is  calculated 
by  convolving  a dilated  wavelet  with  the  original  signal.  The 
wavelet  transform  of  a function  f(x)  at  the  scale  s and  position  x, 
calculated  with  respect  to  the  wavelet  t|/a(x).  is  defined  in  [15] 
as, 


wsaf(x)=/^:(x).  o) 

Similarly,  the  transform  with  respect  to  tpb(x)  is. 

Wbf(x)  = f*y/b(x).  (4) 


The  above  wavelet  transforms  arc  the  first  and  second  derivative 
of  the  signal  smoothed  at  the  scale  or  resolution  level  ,s. 
Substituting  into  (3)  and  (4)  equation  (2)  for  the  1-D  case. 
Mallat  then  derives  a 2-D  expression  for  the  wavelet  transform 
of  a function  or  image. 


= sV{f*OXx,y). 


The  above  wavelet  transform  definitions  in  (5)  are  important  for 
a wavelet  based  clutter  metric  because  they  essentially  define 
edge  detectors  that  arc  used  in  the  vision  science  community. 
For  more  discussion  on  this  topic  see  ref.  [16].  An 
implementation  of  eq.  (5)  in  the  program  XWAVF,  was  used  to 
compute  edge  points. 


4.  IMPLEMF.NTATION 


The  Fuzzy  Inference  System  (FIS)  that  models  the  relationships 
between  the  various  input  variables  that  affect  the  determination 
of  the  search  time  is  done  specifically  for  this  dataset.  The 
predicted  search  time  for  target  detection  can  be  determined 
with  the  FLA  using  input  target  metrics  for  the  images  shown 
below  in  Fig.'s  1 through  6.  The  input  variables  were;  distance 
from  the  target  to  the  observer  (km),  the  aspect  angle  of  the 
vehicle  relative  to  the  observer  (deg),  the  target  height  (pixels) 
and  the  target  area  (pixels4),  target  and  the  local  background 
luminance  (cd/m  2).  and  the  wavelet  determined  edge  points  of 
the  scene  as  a measure  of  clutter.  The  one  output  parameter  is 
the  search  time  (secs).  There  were  a total  of  44  digitized  color 
images  along  w ith  the  associated  target  and  background  metrics 
for  the  targets  in  each  picture.  22  images  arc  used  for  training 
and  22  arc  used  for  testing.  Both  the  Mamdani  and  Sugcno  type 
FIS  methods  are  used  and  compared.  The  authors  constructed 
the  FIS's  to  predict  search  times  using  the  MATLAB  Fuzzy 
Logic  Toolbox  [13]. 


For  convenience  the  algorithm  for  computing  the  wavelet  edge 
points  is  summarized  as  follows: 

• Rend  the  input  256  X 256  element  matrix 
which  supports  a discrete  2-D  image  f(x.y) 

• Take  the  wavelet  transform  of  the  image  (15)  at 
a certain  resolution  level 

• Set  the  threshold,  here  chosen  as  zero 

• Determine  the  number  of  edge  points  in  along 
with  the  number  of  pixels 

• Find  the  edge  density  from  the  number  of  edge 
points  divided  by  the  total  number  of  pixels 

• Iterate  s,  the  level  of  wavelet  in  the  analysis 

• Find  the  probability  of  detection  (Pd)  for  the 
target  in  the  scene. 


Table  I below  lists  the  metrics  used  in  the  trials.  The  table 
entries,  all  except  "Edge  points',  were  provided  by  Dr.  Alex 
Toet  of  TNO.  The  entries  are:  target  type  number,  distance  from 
target  to  sensor,  the  absolute  value  of  the  sin  of  the  aspect  angle 
of  the  vehicle  relative  to  the  observer,  the  height  of  the  target  in 
pixels,  the  area  of  the  target  in  pixels,  the  target  luminance,  the 
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darkest  part  of  the  target  luminance,  the  surrounding  area  output.  Fig.  8 is  the  firing  array  for  the  various  membership 

average  luminance,  edge  points  and  the  mean  search  time  in  functions  using  the  Mandani  approach. 

seconds.  The  edge  points  were  found  using  a wavelet  program 

to  compute  the  number  of  wavelet  edge  points  over  the  whole 

image  to  give  a measure  of  the  clutter  in  the  image. 

Below  in  Fig.  7 is  the  Mamdani  type  FIS  with  the  input 
parameters  mentioned  above  and  the  search  time  as  the  single 


Sample  Visual  Images 

Courtesy  of  Dr.  Alex  Toet  ofTNO 
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Fig.  8 Firing  diagrams  for  the  Mamdani  FIS  to  predict  search  times 
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5.  RESULTS 

Fig.  9 shows  the  correlation  of  laboratory  search  times  to  FLA 
predicted  search  times  using  the  Mamdani  approach  with 
membership  functions  we  designed  and  achieved  a 0.957 
correlation  of  model  predicted  search  times  to  experimental 
search  times.  Fig.  10  is  the  output  of  the  ANFIS  model  of  the 
data,  which  gave  a 0.60  correlation  to  the  data.  We  also  tried 
using  the  Mamdani  FIS  , with  the  0.957  correlation  to 
experiment,  on  another  data  set  of  visual  imagery  [14],  The  FIS 
from  one  data  set  can  be  used  to  model  another  data  set,  if  and 
only  if,  the  metrics  used  to  describe  the  various  data  sets  are 
similar. 


number  of  input  vectors  to  a small  number  to  reduce  the  number 
of  rules  and  membership  functions  that  need  to  be  constructed. 
Clustering  was  used  to  obtain  the  means  of  the  7 input  vectors. 
The  center  of  the  clusters  was  used  in  the  construction  of  the 
membership  functions.  The  correlation  results  are  shown  below 
in  Table  2.  Clusters  were  made  of  1 5,  18  and  20  data  points. 
The  FLA  with  clustering  was  used  to  predict  search  time  for  the 
22  points  not  used  in  obtaining  the  clusters  and  for  the  entire 
data  set  of  44  images. 


TABLE  2 System  Evaluation  Using  Cluster  Centers 


These  results  are  indicative  of  the  power  of  using  the  FLA  to 
model  highly  complex  data,  for  which  there  would  be  many 
interrelated  equations  if  one  tried  to  model  the  detection 
problem  in  the  conventional  standard  algorithm  based  method. 


Predicted  vs.  experimental  search  times  —4— exp",  search  time  (s) 

using  testing  data  » predicted  search  time(s) 

(0.957  correlation)  


Fig.  9 Graph  of  search  times  from  Mamdani  FLA  model 
and  the  laboratory 


Fig.  10  Chart  showing  the  comparison  of  experimental 
search  times  to  ANFIS  FLA  predicted  search  times 
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It  is  expected  that  increasing  the  number  of  cluster  centers  and 
the  number  of  rules  will  improve  the  correlation.  This  is  not  the 
case  when  the  number  of  clusters  was  increased  from  15  to  18. 
The  reason  for  this  is  due  to  the  random  operations  used  in 
clusters’  center  calculations.  In  other  words,  if  we  started  from 
another  clusters’  center  we  may  get  better  correlation.  We  used 
the  cluster  centers  as  the  centers  of  membership  functions,  but 
chose  initial  values  for  the  width.  We  can  then  tune  the  width 
manually  to  increase  the  correlation.  It  is  clear  that  there  needs 
to  be  an  objective  algorithm  or  technique  to  tune  the  width  of  the 
membership  functions  as  ANFIS  does.  Below  in  Fig.  11  is  a 
snapshot  of  the  result  of  clustering  the  input  variable  distance 
over  15  cluster  means. 


4 Clustering 


£iie  £dit  Window  Help 


Clustering  was  also  used  to  model  the  visual  metrics  and 
responses.  For  a large  dataset,  it  will  be  desirable  to  reduce  the 


Fig.  11  Clustering  results  using  15  clusters 
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6.  CONCLUSION 

In  conclusion,  the  FLA  yields  very  satisfactorily  results.  0.97 
correlation  of  laboratory  or  field  data  to  model  predicted  data, 
and  requires  a fraction  of  the  effort  that  goes  into  traditional 
algorithm  based  techniques  of  modeling  target  acquisition 
probabilities  and  search  times.  We  expect  that  the  fuzzy 
modeling  approach  could  be  used  in  the  existing  statistical 
decision  theory  modules  of  target  acquisition  models  for  any 
spectral  regime. 

Two  fuzzy  models  have  been  used:  namely  the  Mamdani  and 
Sugeno  models.  This  application  of  the  FLA  involved  pictures, 
metrics,  and  experimental  search  times  of  images  in  the  visual 
band.  Future  work  will  involve  the  application  of  the  FLA  to 
predict  the  Pd’s  of  moving  targets  in  visual  and  infrared 
cluttered  scenes  for  military  and  commercial  applications. 
Clustering  of  the  input  data  was  explored  as  a means  to  reduce 
the  number  of  input  vectors  and  membership  functions.  For 
large  data  sets,  a saving  of  computational  time  and  effort  should 
be  realized  using  this  approach.  The  membership  functions  can 
be  designed  using  experimental  Pel's  or  search  times  collected  in 
the  TARDF.C  Visual  Perception  Laboratory  (VPL).  The 
TARDFC  VPL  is  being  used  in  a collaborative  R&D  project 
with  auto  companies  on  vehicle  conspicuity. 
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